Simulated Dopamine Modulation of a Neurorobotic Model of the Basal Ganglia.

The vertebrate basal ganglia play an important role in action selection-the resolution of conflicts between alternative motor programs. The effective operation of basal ganglia circuitry is also known to rely on appropriate levels of the neurotransmitter dopamine. We investigated reducing or increasing the tonic level of simulated dopamine in a prior model of the basal ganglia integrated into a robot control architecture engaged in a foraging task inspired by animal behaviour. The main findings were that progressive reductions in the levels of simulated dopamine caused slowed behaviour and, at low levels, an inability to initiate movement. These states were partially relieved by increased salience levels (stronger sensory/motivational input). Conversely, increased simulated dopamine caused distortion of the robot's motor acts through partially expressed motor activity relating to losing actions. This could also lead to an increased frequency of behaviour switching. Levels of simulated dopamine that were either significantly lower or higher than baseline could cause a loss of behavioural integration, sometimes leaving the robot in a 'behavioral trap'. That some analogous traits are observed in animals and humans affected by dopamine dysregulation suggests that robotic models could prove useful in understanding the role of dopamine neurotransmission in basal ganglia function and dysfunction.

Tracking subjects' strategies in behavioural choice experiments at trial resolution.

Investigating how, when, and what subjects learn during decision-making tasks requires tracking their choice strategies on a trial-by-trial basis. Here, we present a simple but effective probabilistic approach to tracking choice strategies at trial resolution using Bayesian evidence accumulation. We show this approach identifies both successful learning and the exploratory strategies used in decision tasks performed by humans, non-human primates, rats, and synthetic agents. Both when subjects learn and when rules change the exploratory strategies of win-stay and lose-shift, often considered complementary, are consistently used independently. Indeed, we find the use of lose-shift is strong evidence that subjects have latently learnt the salient features of a new rewarded rule. Our approach can be extended to any discrete choice strategy, and its low computational cost is ideally suited for real-time analysis and closed-loop control.

Motor cortex latent dynamics encode arm movement direction and urgency independently.

The fluid movement of an arm is controlled by multiple parameters that can be set independently. Recent studies argue that arm movements are generated by the collective dynamics of neurons in motor cortex. An untested prediction of this hypothesis is that independent parameters of movement must map to independently-specifiable dynamics. Using a task where monkeys made sequential, varied arm movements, we show that independent parameters of arm movements are independently encoded in the low-dimensional trajectories of population activity: each movement's direction by a fixed neural trajectory and its urgency by how quickly that trajectory was traversed. Network models show this latent coding allows the direction and urgency of arm movement to be independently controlled. Our results support a key prediction of the dynamical systems view of motor cortex, but also argue that not all parameters of movement are defined by the initial conditions of those dynamics.

Activity Subspaces in Medial Prefrontal Cortex Distinguish States of the World.

Medial prefrontal cortex (mPfC) activity represents information about the state of the world, including present behavior, such as decisions, and the immediate past, such as short-term memory. Unknown is whether information about different states of the world are represented in the same mPfC neural population and, if so, how they are kept distinct. To address this, we analyze here mPfC population activity of male rats learning rules in a Y-maze, with self-initiated choice trials to an arm end followed by a self-paced return during the intertrial interval (ITI). We find that trial and ITI population activity from the same population fall into different low-dimensional subspaces. These subspaces encode different states of the world: multiple features of the task can be decoded from both trial and ITI activity, but the decoding axes for the same feature are roughly orthogonal between the two task phases, and the decodings are predominantly of features of the present during the trial but features of the preceding trial during the ITI. These subspace distinctions are carried forward into sleep, where population activity is preferentially reactivated in post-training sleep but differently for activity from the trial and ITI subspaces. Our results suggest that the problem of interference when representing different states of the world is solved in mPfC by population activity occupying different subspaces for the world states, which can be independently decoded by downstream targets and independently addressed by upstream inputs.SIGNIFICANCE STATEMENT Activity in the medial prefrontal cortex plays a role in representing the current and past states of the world. We show that during a maze task, the activity of a single population in medial prefrontal cortex represents at least two different states of the world. These representations were sequential and sufficiently distinct that a downstream population could separately read out either state from that activity. Moreover, the activity representing different states is differently reactivated in sleep. Different world states can thus be represented in the same medial prefrontal cortex population but in such a way that prevents potentially catastrophic interference between them.

Bayesian Mapping of the Striatal Microcircuit Reveals Robust Asymmetries in the Probabilities and Distances of Connections.

The striatum's complex microcircuit is made by connections within and between its D1- and D2-receptor expressing projection neurons and at least five species of interneuron. Precise knowledge of this circuit is likely essential to understanding striatum's functional roles and its dysfunction in a wide range of movement and cognitive disorders. We introduce here a Bayesian approach to mapping neuron connectivity using intracellular recording data, which lets us simultaneously evaluate the probability of connection between neuron types, the strength of evidence for it, and its dependence on distance. Using it to synthesize a complete map of the mouse striatum, we find strong evidence for two asymmetries: a selective asymmetry of projection neuron connections, with D2 neurons connecting twice as densely to other projection neurons than do D1 neurons, but neither subtype preferentially connecting to another; and a length-scale asymmetry, with interneuron connection probabilities remaining non-negligible at more than twice the distance of projection neuron connections. We further show that our Bayesian approach can evaluate evidence for wiring changes, using data from the developing striatum and a mouse model of Huntington's disease. By quantifying the uncertainty in our knowledge of the microcircuit, our approach reveals a wide range of potential striatal wiring diagrams consistent with current data.SIGNIFICANCE STATEMENT To properly understand a neuronal circuit's function, it is important to have an accurate picture of the rate of connection between individual neurons and how this rate changes with the distance separating pairs of neurons. We present a Bayesian method for extracting this information from experimental data and apply it to the mouse striatum, a subcortical structure involved in learning and decision-making, which is made up of a variety of different projection neurons and interneurons. Our resulting statistical map reveals not just the most robust estimates of the probability of connection between neuron types, but also the strength of evidence for them, and their dependence on distance.

Making decisions in the dark basement of the brain: A look back at the GPR model of action selection and the basal ganglia.

How does your brain decide what you will do next? Over the past few decades compelling evidence has emerged that the basal ganglia, a collection of nuclei in the fore- and mid-brain of all vertebrates, are vital to action selection. Gurney, Prescott, and Redgrave published an influential computational account of this idea in Biological Cybernetics in 2001. Here we take a look back at this pair of papers, outlining the "GPR" model contained therein, the context of that model's development, and the influence it has had over the past twenty years. Tracing its lineage into models and theories still emerging now, we are encouraged that the GPR model is that rare thing, a computational model of a brain circuit whose advances were directly built on by others.

Spectral estimation for detecting low-dimensional structure in networks using arbitrary null models.

Discovering low-dimensional structure in real-world networks requires a suitable null model that defines the absence of meaningful structure. Here we introduce a spectral approach for detecting a network's low-dimensional structure, and the nodes that participate in it, using any null model. We use generative models to estimate the expected eigenvalue distribution under a specified null model, and then detect where the data network's eigenspectra exceed the estimated bounds. On synthetic networks, this spectral estimation approach cleanly detects transitions between random and community structure, recovers the number and membership of communities, and removes noise nodes. On real networks spectral estimation finds either a significant fraction of noise nodes or no departure from a null model, in stark contrast to traditional community detection methods. Across all analyses, we find the choice of null model can strongly alter conclusions about the presence of network structure. Our spectral estimation approach is therefore a promising basis for detecting low-dimensional structure in real-world networks, or lack thereof.

Prediction of Choice from Competing Mechanosensory and Choice-Memory Cues during Active Tactile Decision Making.

Perceptual decision making is an active process where animals move their sense organs to extract task-relevant information. To investigate how the brain translates sensory input into decisions during active sensation, we developed a mouse active touch task where the mechanosensory input can be precisely measured and that challenges animals to use multiple mechanosensory cues. Male mice were trained to localize a pole using a single whisker and to report their decision by selecting one of three choices. Using high-speed imaging and machine vision, we estimated whisker-object mechanical forces at millisecond resolution. Mice solved the task by a sensory-motor strategy where both the strength and direction of whisker bending were informative cues to pole location. We found competing influences of immediate sensory input and choice memory on mouse choice. On correct trials, choice could be predicted from the direction and strength of whisker bending, but not from previous choice. In contrast, on error trials, choice could be predicted from previous choice but not from whisker bending. This study shows that animal choices during active tactile decision making can be predicted from mechanosensory and choice-memory signals, and provides a new task well suited for the future study of the neural basis of active perceptual decisions.SIGNIFICANCE STATEMENT Due to the difficulty of measuring the sensory input to moving sense organs, active perceptual decision making remains poorly understood. The whisker system provides a way forward since it is now possible to measure the mechanical forces due to whisker-object contact during behavior. Here we train mice in a novel behavioral task that challenges them to use rich mechanosensory cues but can be performed using one whisker and enables task-relevant mechanical forces to be precisely estimated. This approach enables rigorous study of how sensory cues translate into action during active, perceptual decision making. Our findings provide new insight into active touch and how sensory/internal signals interact to determine behavioral choices.

Medial Prefrontal Cortex Population Activity Is Plastic Irrespective of Learning.

The prefrontal cortex (PFC) is thought to learn the relationships between actions and their outcomes. But little is known about what changes to population activity in PFC are specific to learning these relationships. Here we characterize the plasticity of population activity in the medial PFC (mPFC) of male rats learning rules on a Y-maze. First, we show that the population always changes its patterns of joint activity between the periods of sleep either side of a training session on the maze, regardless of successful rule learning during training. Next, by comparing the structure of population activity in sleep and training, we show that this population plasticity differs between learning and nonlearning sessions. In learning sessions, the changes in population activity in post-training sleep incorporate the changes to the population activity during training on the maze. In nonlearning sessions, the changes in sleep and training are unrelated. Finally, we show evidence that the nonlearning and learning forms of population plasticity are driven by different neuron-level changes, with the nonlearning form entirely accounted for by independent changes to the excitability of individual neurons, and the learning form also including changes to firing rate couplings between neurons. Collectively, our results suggest two different forms of population plasticity in mPFC during the learning of action-outcome relationships: one a persistent change in population activity structure decoupled from overt rule-learning, and the other a directional change driven by feedback during behavior.SIGNIFICANCE STATEMENT The PFC is thought to represent our knowledge about what action is worth doing in which context. But we do not know how the activity of neurons in PFC collectively changes when learning which actions are relevant. Here we show, in a trial-and-error task, that population activity in PFC is persistently changing, regardless of learning. Only during episodes of clear learning of relevant actions are the accompanying changes to population activity carried forward into sleep, suggesting a long-lasting form of neural plasticity. Our results suggest that representations of relevant actions in PFC are acquired by reward imposing a direction onto ongoing population plasticity.

Dynamical networks: Finding, measuring, and tracking neural population activity using network science.

Systems neuroscience is in a headlong rush to record from as many neurons at the same time as possible. As the brain computes and codes using neuron populations, it is hoped these data will uncover the fundamentals of neural computation. But with hundreds, thousands, or more simultaneously recorded neurons come the inescapable problems of visualizing, describing, and quantifying their interactions. Here I argue that network science provides a set of scalable, analytical tools that already solve these problems. By treating neurons as nodes and their interactions as links, a single network can visualize and describe an arbitrarily large recording. I show that with this description we can quantify the effects of manipulating a neural circuit, track changes in population dynamics over time, and quantitatively define theoretical concepts of neural populations such as cell assemblies. Using network science as a core part of analyzing population recordings will thus provide both qualitative and quantitative advances to our understanding of neural computation.

An ensemble code in medial prefrontal cortex links prior events to outcomes during learning.

The prefrontal cortex is implicated in learning the rules of an environment through trial and error. But it is unclear how such learning is related to the prefrontal cortex's role in short-term memory. Here we ask if the encoding of short-term memory in prefrontal cortex is used by rats learning decision rules in a Y-maze task. We find that a similar pattern of neural ensemble activity is selectively recalled after reinforcement for a correct decision. This reinforcement-selective recall only reliably occurs immediately before the abrupt behavioural transitions indicating successful learning of the current rule, and fades quickly thereafter. We could simultaneously decode multiple, retrospective task events from the ensemble activity, suggesting the recalled ensemble activity has multiplexed encoding of prior events. Our results suggest that successful trial-and-error learning is dependent on reinforcement tagging the relevant features of the environment to maintain in prefrontal cortex short-term memory.

A probabilistic, distributed, recursive mechanism for decision-making in the brain.

Decision formation recruits many brain regions, but the procedure they jointly execute is unknown. Here we characterize its essential composition, using as a framework a novel recursive Bayesian algorithm that makes decisions based on spike-trains with the statistics of those in sensory cortex (MT). Using it to simulate the random-dot-motion task, we demonstrate it quantitatively replicates the choice behaviour of monkeys, whilst predicting losses of otherwise usable information from MT. Its architecture maps to the recurrent cortico-basal-ganglia-thalamo-cortical loops, whose components are all implicated in decision-making. We show that the dynamics of its mapped computations match those of neural activity in the sensorimotor cortex and striatum during decisions, and forecast those of basal ganglia output and thalamus. This also predicts which aspects of neural dynamics are and are not part of inference. Our single-equation algorithm is probabilistic, distributed, recursive, and parallel. Its success at capturing anatomy, behaviour, and electrophysiology suggests that the mechanism implemented by the brain has these same characteristics.

Insights into Parkinson's disease from computational models of the basal ganglia.

Movement disorders arise from the complex interplay of multiple changes to neural circuits. Successful treatments for these disorders could interact with these complex changes in myriad ways, and as a consequence their mechanisms of action and their amelioration of symptoms are incompletely understood. Using Parkinson's disease as a case study, we review here how computational models are a crucial tool for taming this complexity, across causative mechanisms, consequent neural dynamics and treatments. For mechanisms, we review models that capture the effects of losing dopamine on basal ganglia function; for dynamics, we discuss models that have transformed our understanding of how beta-band (15-30 Hz) oscillations arise in the parkinsonian basal ganglia. For treatments, we touch on the breadth of computational modelling work trying to understand the therapeutic actions of deep brain stimulation. Collectively, models from across all levels of description are providing a compelling account of the causes, symptoms and treatments for Parkinson's disease.

A spiral attractor network drives rhythmic locomotion.

The joint activity of neural populations is high dimensional and complex. One strategy for reaching a tractable understanding of circuit function is to seek the simplest dynamical system that can account for the population activity. By imaging Aplysia's pedal ganglion during fictive locomotion, here we show that its population-wide activity arises from a low-dimensional spiral attractor. Evoking locomotion moved the population into a low-dimensional, periodic, decaying orbit - a spiral - in which it behaved as a true attractor, converging to the same orbit when evoked, and returning to that orbit after transient perturbation. We found the same attractor in every preparation, and could predict motor output directly from its orbit, yet individual neurons' participation changed across consecutive locomotion bouts. From these results, we propose that only the low-dimensional dynamics for movement control, and not the high-dimensional population activity, are consistent within and between nervous systems.

The Goldilocks zone in neural circuits.

How do networks of neurons remain both stable and sensitive to new inputs?

Finding communities in sparse networks.

Spectral algorithms based on matrix representations of networks are often used to detect communities, but classic spectral methods based on the adjacency matrix and its variants fail in sparse networks. New spectral methods based on non-backtracking random walks have recently been introduced that successfully detect communities in many sparse networks. However, the spectrum of non-backtracking random walks ignores hanging trees in networks that can contain information about their community structure. We introduce the reluctant backtracking operators that explicitly account for hanging trees as they admit a small probability of returning to the immediately previous node, unlike the non-backtracking operators that forbid an immediate return. We show that the reluctant backtracking operators can detect communities in certain sparse networks where the non-backtracking operators cannot, while performing comparably on benchmark stochastic block model networks and real world networks. We also show that the spectrum of the reluctant backtracking operator approximately optimises the standard modularity function. Interestingly, for this family of non- and reluctant-backtracking operators the main determinant of performance on real-world networks is whether or not they are normalised to conserve probability at each node.

Modular deconstruction reveals the dynamical and physical building blocks of a locomotion motor program.

The neural substrates of motor programs are only well understood for small, dedicated circuits. Here we investigate how a motor program is constructed within a large network. We imaged populations of neurons in the Aplysia pedal ganglion during execution of a locomotion motor program. We found that the program was built from a very small number of dynamical building blocks, including both neural ensembles and low-dimensional rotational dynamics. These map onto physically discrete regions of the ganglion, so that the motor program has a corresponding modular organization in both dynamical and physical space. Using this dynamic map, we identify the population potentially implementing the rhythmic pattern generator and find that its activity physically traces a looped trajectory, recapitulating its low-dimensional rotational dynamics. Our results suggest that, even in simple invertebrates, neural motor programs are implemented by large, distributed networks containing multiple dynamical systems.

A new framework for cortico-striatal plasticity: behavioural theory meets in vitro data at the reinforcement-action interface.

Operant learning requires that reinforcement signals interact with action representations at a suitable neural interface. Much evidence suggests that this occurs when phasic dopamine, acting as a reinforcement prediction error, gates plasticity at cortico-striatal synapses, and thereby changes the future likelihood of selecting the action(s) coded by striatal neurons. But this hypothesis faces serious challenges. First, cortico-striatal plasticity is inexplicably complex, depending on spike timing, dopamine level, and dopamine receptor type. Second, there is a credit assignment problem-action selection signals occur long before the consequent dopamine reinforcement signal. Third, the two types of striatal output neuron have apparently opposite effects on action selection. Whether these factors rule out the interface hypothesis and how they interact to produce reinforcement learning is unknown. We present a computational framework that addresses these challenges. We first predict the expected activity changes over an operant task for both types of action-coding striatal neuron, and show they co-operate to promote action selection in learning and compete to promote action suppression in extinction. Separately, we derive a complete model of dopamine and spike-timing dependent cortico-striatal plasticity from in vitro data. We then show this model produces the predicted activity changes necessary for learning and extinction in an operant task, a remarkable convergence of a bottom-up data-driven plasticity model with the top-down behavioural requirements of learning theory. Moreover, we show the complex dependencies of cortico-striatal plasticity are not only sufficient but necessary for learning and extinction. Validating the model, we show it can account for behavioural data describing extinction, renewal, and reacquisition, and replicate in vitro experimental data on cortico-striatal plasticity. By bridging the levels between the single synapse and behaviour, our model shows how striatum acts as the action-reinforcement interface.

Early hypersynchrony in juvenile PINK1(-)/(-) motor cortex is rescued by antidromic stimulation.

In Parkinson's disease (PD), cortical networks show enhanced synchronized activity but whether this precedes motor signs is unknown. We investigated this question in PINK1(-)/(-) mice, a genetic rodent model of the PARK6 variant of familial PD which shows impaired spontaneous locomotion at 16 months. We used two-photon calcium imaging and whole-cell patch clamp in slices from juvenile (P14-P21) wild-type or PINK1(-)/(-) mice. We designed a horizontal tilted cortico-subthalamic slice where the only connection between cortex and subthalamic nucleus (STN) is the hyperdirect cortico-subthalamic pathway. We report excessive correlation and synchronization in PINK1(-)/(-) M1 cortical networks 15 months before motor impairment. The percentage of correlated pairs of neurons and their strength of correlation were higher in the PINK1(-)/(-) M1 than in the wild type network and the synchronized network events involved a higher percentage of neurons. Both features were independent of thalamo-cortical pathways, insensitive to chronic levodopa treatment of pups, but totally reversed by antidromic invasion of M1 pyramidal neurons by axonal spikes evoked by high frequency stimulation (HFS) of the STN. Our study describes an early excess of synchronization in the PINK1(-)/(-) cortex and suggests a potential role of antidromic activation of cortical interneurons in network desynchronization. Such backward effect on interneurons activity may be of importance for HFS-induced network desynchronization.